Skip to content

[fix](virtual slot) Fix grouping scalar function #56127

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
englefly merged 2 commits into from
Sep 17, 2025
Merged

[fix](virtual slot) Fix grouping scalar function #56127

englefly merged 2 commits into from
Sep 17, 2025

Conversation

@zhiqiang-hhhh
Copy link
Contributor

@zhiqiang-hhhh zhiqiang-hhhh commented Sep 17, 2025 •
edited by englefly
Loading

What problem does this PR solve?

Related PR: #52701

  1. Do not optimize grouping scalar function.
  2. Fix rule type of ann topn push down.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Sep 17, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@zhiqiang-hhhh
Copy link
Contributor Author

zhiqiang-hhhh commented Sep 17, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Sep 17, 2025

TPC-H: Total hot run time: 34960 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1529c5ed789a6fc362a63f25ad850e436ccaefbc, data reload: false

------ Round 1 ----------------------------------
q1	17626	5215	5109	5109
q2	2017	317	208	208
q3	10252	1336	745	745
q4	10225	1040	550	550
q5	7585	2377	2345	2345
q6	192	165	135	135
q7	951	769	674	674
q8	9360	1285	1138	1138
q9	6967	5135	5145	5135
q10	6897	2399	1964	1964
q11	489	304	297	297
q12	346	377	234	234
q13	17774	3675	3109	3109
q14	265	238	234	234
q15	567	517	495	495
q16	1021	1001	950	950
q17	601	871	376	376
q18	7475	7281	7083	7083
q19	1238	947	554	554
q20	342	334	229	229
q21	3736	3181	2414	2414
q22	1102	1024	982	982
Total cold run time: 107028 ms
Total hot run time: 34960 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5093	5089	5088	5088
q2	251	343	238	238
q3	2137	2696	2328	2328
q4	1357	1757	1343	1343
q5	4231	4559	4610	4559
q6	229	175	129	129
q7	2102	2007	1812	1812
q8	2643	2790	2607	2607
q9	7489	7348	7462	7348
q10	3121	3315	2952	2952
q11	583	521	513	513
q12	676	839	657	657
q13	3531	4039	3395	3395
q14	294	334	288	288
q15	550	486	494	486
q16	1098	1138	1046	1046
q17	1194	1585	1394	1394
q18	7965	7707	7506	7506
q19	837	825	976	825
q20	2017	2077	1878	1878
q21	5049	4462	4278	4278
q22	1106	1049	1013	1013
Total cold run time: 53553 ms
Total hot run time: 51683 ms

@doris-robot
Copy link

doris-robot commented Sep 17, 2025

TPC-DS: Total hot run time: 189146 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1529c5ed789a6fc362a63f25ad850e436ccaefbc, data reload: false

query1	1084	449	410	410
query2	6565	1674	1724	1674
query3	6746	224	221	221
query4	26387	23290	23356	23290
query5	4526	618	495	495
query6	346	253	241	241
query7	4680	514	305	305
query8	308	281	254	254
query9	8669	2637	2631	2631
query10	498	333	285	285
query11	16126	15066	14845	14845
query12	171	113	111	111
query13	1670	555	437	437
query14	11497	9266	9273	9266
query15	215	191	191	191
query16	7686	699	549	549
query17	1260	740	621	621
query18	2050	431	332	332
query19	201	202	172	172
query20	190	122	135	122
query21	226	146	124	124
query22	4072	4363	4157	4157
query23	33956	33009	33067	33009
query24	8387	2408	2417	2408
query25	563	510	438	438
query26	1234	275	160	160
query27	2739	510	361	361
query28	4360	2231	2219	2219
query29	768	612	488	488
query30	295	226	199	199
query31	933	812	765	765
query32	81	80	72	72
query33	597	375	336	336
query34	814	861	542	542
query35	803	832	741	741
query36	998	1047	949	949
query37	133	112	84	84
query38	3554	3556	3556	3556
query39	1505	1428	1410	1410
query40	220	138	125	125
query41	70	70	66	66
query42	139	122	118	118
query43	517	494	475	475
query44	1369	872	866	866
query45	196	183	216	183
query46	870	1041	654	654
query47	1750	1804	1718	1718
query48	392	432	320	320
query49	766	502	435	435
query50	694	693	413	413
query51	3889	4038	3806	3806
query52	119	116	105	105
query53	249	268	201	201
query54	612	601	564	564
query55	89	86	87	86
query56	340	313	297	297
query57	1207	1179	1131	1131
query58	278	268	268	268
query59	2585	2684	2605	2605
query60	355	339	323	323
query61	181	169	166	166
query62	840	738	655	655
query63	233	197	195	195
query64	4451	1144	810	810
query65	4023	3964	3984	3964
query66	1083	428	365	365
query67	15682	15355	15358	15355
query68	8013	924	600	600
query69	480	324	297	297
query70	1335	1338	1266	1266
query71	594	333	316	316
query72	5853	5047	4976	4976
query73	622	619	362	362
query74	9065	9181	8637	8637
query75	3548	3312	2818	2818
query76	3317	1198	745	745
query77	575	407	316	316
query78	9702	9669	8848	8848
query79	2681	844	601	601
query80	697	571	523	523
query81	509	270	237	237
query82	395	160	130	130
query83	273	266	250	250
query84	254	189	100	100
query85	897	464	416	416
query86	376	320	312	312
query87	3741	3786	3691	3691
query88	3895	2248	2253	2248
query89	410	340	294	294
query90	1951	226	214	214
query91	171	175	149	149
query92	91	68	69	68
query93	2609	1015	638	638
query94	689	412	353	353
query95	407	319	306	306
query96	483	579	275	275
query97	2920	2992	2888	2888
query98	249	218	210	210
query99	1357	1425	1323	1323
Total cold run time: 278251 ms
Total hot run time: 189146 ms

@doris-robot
Copy link

doris-robot commented Sep 17, 2025

ClickBench: Total hot run time: 29.67 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1529c5ed789a6fc362a63f25ad850e436ccaefbc, data reload: false

query1	0.06	0.05	0.04
query2	0.09	0.06	0.06
query3	0.25	0.08	0.08
query4	1.61	0.11	0.11
query5	0.28	0.27	0.25
query6	1.18	0.66	0.65
query7	0.03	0.03	0.03
query8	0.06	0.05	0.04
query9	0.63	0.52	0.54
query10	0.57	0.58	0.56
query11	0.16	0.12	0.11
query12	0.16	0.12	0.12
query13	0.63	0.64	0.61
query14	1.02	1.02	1.05
query15	0.85	0.85	0.86
query16	0.39	0.40	0.40
query17	1.08	1.06	1.01
query18	0.25	0.20	0.20
query19	1.92	1.79	1.83
query20	0.02	0.02	0.01
query21	15.43	0.97	0.58
query22	0.77	1.22	0.68
query23	14.89	1.41	0.66
query24	6.68	1.69	0.50
query25	0.45	0.25	0.08
query26	0.61	0.16	0.14
query27	0.06	0.06	0.06
query28	9.19	0.88	0.45
query29	12.65	3.93	3.25
query30	0.28	0.13	0.11
query31	2.83	0.59	0.41
query32	3.26	0.56	0.47
query33	3.08	3.10	3.24
query34	16.17	5.45	4.98
query35	4.94	4.92	4.88
query36	0.71	0.52	0.50
query37	0.10	0.08	0.07
query38	0.07	0.05	0.04
query39	0.04	0.02	0.03
query40	0.18	0.16	0.14
query41	0.08	0.03	0.03
query42	0.04	0.03	0.04
query43	0.05	0.03	0.04
Total cold run time: 103.8 s
Total hot run time: 29.67 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 17, 2025

FE UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 17, 2025

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

@zhiqiang-hhhh
Copy link
Contributor Author

zhiqiang-hhhh commented Sep 17, 2025

run buildall

@zhiqiang-hhhh zhiqiang-hhhh changed the title [fix](ann topn rule) Fix rule type of ann topn push down [fix](virtual slot) Fix grouping scalar function Sep 17, 2025
@doris-robot
Copy link

doris-robot commented Sep 17, 2025

TPC-H: Total hot run time: 35158 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9947ac5f80de2e3adaf6cbb67a0411c3fc82c34e, data reload: false

------ Round 1 ----------------------------------
q1	17629	5272	5141	5141
q2	2014	313	223	223
q3	10248	1302	768	768
q4	10238	1068	540	540
q5	7544	2470	2427	2427
q6	193	169	142	142
q7	956	769	655	655
q8	9372	1305	1199	1199
q9	7010	5111	5143	5111
q10	6915	2407	1983	1983
q11	494	299	290	290
q12	353	389	238	238
q13	17794	3659	3062	3062
q14	251	244	223	223
q15	557	531	487	487
q16	1030	999	949	949
q17	606	873	353	353
q18	7686	7149	6959	6959
q19	1248	949	609	609
q20	355	345	238	238
q21	3945	3318	2573	2573
q22	1081	1044	988	988
Total cold run time: 107519 ms
Total hot run time: 35158 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5174	5187	5097	5097
q2	263	344	234	234
q3	2218	2685	2320	2320
q4	1395	1791	1348	1348
q5	4269	4672	4580	4580
q6	232	189	137	137
q7	2029	2046	1854	1854
q8	2704	2670	2630	2630
q9	7347	7338	7256	7256
q10	3187	3294	2904	2904
q11	596	530	535	530
q12	696	794	652	652
q13	3736	3919	3204	3204
q14	332	330	323	323
q15	535	493	507	493
q16	1091	1116	1053	1053
q17	1208	1589	1364	1364
q18	7828	7721	7665	7665
q19	784	871	1018	871
q20	2031	2104	2015	2015
q21	5234	4637	4323	4323
q22	1104	1066	996	996
Total cold run time: 53993 ms
Total hot run time: 51849 ms

@doris-robot
Copy link

doris-robot commented Sep 17, 2025

TPC-DS: Total hot run time: 188395 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9947ac5f80de2e3adaf6cbb67a0411c3fc82c34e, data reload: false

query1	1055	442	416	416
query2	6565	1728	1757	1728
query3	6755	217	219	217
query4	26536	24079	22877	22877
query5	5614	626	477	477
query6	348	256	240	240
query7	4676	534	310	310
query8	307	262	281	262
query9	8703	2646	2639	2639
query10	556	353	327	327
query11	15250	15053	14952	14952
query12	173	122	114	114
query13	1686	601	439	439
query14	11171	9359	9237	9237
query15	248	200	190	190
query16	7785	727	532	532
query17	1627	782	645	645
query18	2062	445	382	382
query19	265	198	167	167
query20	126	127	124	124
query21	220	136	114	114
query22	4225	4203	4133	4133
query23	34014	33117	32794	32794
query24	8437	2470	2430	2430
query25	544	507	436	436
query26	1229	275	159	159
query27	2692	506	353	353
query28	4394	2254	2229	2229
query29	776	608	489	489
query30	303	223	201	201
query31	902	840	764	764
query32	77	76	67	67
query33	587	389	339	339
query34	799	865	508	508
query35	818	860	752	752
query36	965	1023	914	914
query37	123	103	83	83
query38	3583	3485	3504	3485
query39	1520	1446	1431	1431
query40	222	132	122	122
query41	69	61	62	61
query42	133	114	113	113
query43	525	515	472	472
query44	1323	859	846	846
query45	191	177	170	170
query46	871	1030	655	655
query47	1745	1764	1720	1720
query48	409	425	315	315
query49	780	521	406	406
query50	654	682	402	402
query51	3876	3999	3831	3831
query52	113	106	103	103
query53	238	265	204	204
query54	604	584	529	529
query55	88	85	87	85
query56	317	317	295	295
query57	1205	1214	1101	1101
query58	281	283	269	269
query59	2658	2758	2530	2530
query60	351	347	344	344
query61	171	169	163	163
query62	823	767	687	687
query63	234	196	189	189
query64	4427	1198	818	818
query65	4047	3971	3978	3971
query66	1041	442	356	356
query67	15491	15461	15163	15163
query68	8507	940	592	592
query69	524	382	281	281
query70	1329	1288	1285	1285
query71	590	335	305	305
query72	5897	5051	5049	5049
query73	736	629	365	365
query74	8963	8766	9063	8766
query75	4144	3319	2847	2847
query76	3856	1158	736	736
query77	807	403	315	315
query78	9570	9745	8914	8914
query79	2743	803	592	592
query80	651	556	503	503
query81	471	279	225	225
query82	463	160	133	133
query83	360	268	245	245
query84	313	112	92	92
query85	869	469	423	423
query86	350	319	320	319
query87	3867	3699	3610	3610
query88	3182	2234	2218	2218
query89	438	336	307	307
query90	1971	218	221	218
query91	179	165	140	140
query92	83	67	75	67
query93	1576	984	657	657
query94	682	454	343	343
query95	384	315	307	307
query96	479	573	281	281
query97	2993	3006	2874	2874
query98	244	224	218	218
query99	1430	1399	1292	1292
Total cold run time: 279254 ms
Total hot run time: 188395 ms

@doris-robot
Copy link

doris-robot commented Sep 17, 2025

ClickBench: Total hot run time: 29.84 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9947ac5f80de2e3adaf6cbb67a0411c3fc82c34e, data reload: false

query1	0.06	0.05	0.05
query2	0.08	0.05	0.05
query3	0.26	0.09	0.08
query4	1.61	0.12	0.12
query5	0.28	0.26	0.25
query6	1.16	0.66	0.65
query7	0.03	0.03	0.03
query8	0.06	0.04	0.05
query9	0.64	0.52	0.52
query10	0.58	0.61	0.59
query11	0.18	0.11	0.11
query12	0.16	0.13	0.12
query13	0.64	0.63	0.62
query14	1.02	1.03	1.03
query15	0.87	0.87	0.86
query16	0.40	0.41	0.39
query17	1.02	1.05	1.04
query18	0.22	0.20	0.20
query19	1.97	1.85	1.86
query20	0.02	0.02	0.02
query21	15.46	0.95	0.59
query22	0.76	1.18	0.66
query23	14.98	1.41	0.65
query24	7.31	0.74	0.60
query25	0.51	0.30	0.12
query26	0.55	0.16	0.13
query27	0.07	0.05	0.06
query28	9.91	0.93	0.44
query29	12.56	3.92	3.26
query30	0.28	0.13	0.13
query31	2.82	0.59	0.40
query32	3.24	0.58	0.49
query33	3.04	3.09	3.25
query34	16.09	5.49	4.82
query35	4.99	4.93	4.92
query36	0.71	0.52	0.50
query37	0.10	0.07	0.07
query38	0.06	0.05	0.04
query39	0.04	0.03	0.03
query40	0.18	0.15	0.14
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 105.08 s
Total hot run time: 29.84 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 17, 2025

FE UT Coverage Report

Increment line coverage 57.14% (4/7) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 17, 2025

FE Regression Coverage Report

Increment line coverage 42.86% (3/7) 🎉
Increment coverage report
Complete coverage report

airborne12
airborne12 approved these changes Sep 17, 2025
View reviewed changes
Copy link
Member

@airborne12 airborne12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 17, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Sep 17, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 17, 2025

PR approved by anyone and no changes requested.

@englefly englefly merged commit f05cf95 into apache:master Sep 17, 2025
32 of 34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@englefly englefly englefly approved these changes

@airborne12 airborne12 airborne12 approved these changes

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@zhiqiang-hhhh @Thearas @doris-robot @hello-stephen @englefly @airborne12 @morrySnow